Skip to content

Enable reading Parquet's bloomfilter statistics for Iceberg connector#17192

Merged
raunaqmorarka merged 3 commits intotrinodb:masterfrom
groupcache4321:iceberg_bloomfilter_support
Apr 25, 2023
Merged

Enable reading Parquet's bloomfilter statistics for Iceberg connector#17192
raunaqmorarka merged 3 commits intotrinodb:masterfrom
groupcache4321:iceberg_bloomfilter_support

Conversation

@groupcache4321
Copy link
Copy Markdown
Contributor

@groupcache4321 groupcache4321 commented Apr 23, 2023 •
edited
Loading

Description

Related HIVE PR: #14428 and #15633.
This brings the same change to the Iceberg connector.

Additional context and related issues

Enable hive connector to read parquet file's bloomfilter statistics. For more details: https://github.com/apache/parquet-format/blob/master/BloomFilter.md.

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Iceberg
* Improve performance of queries with filters when bloom filter indexes are present in parquet files. Usage of bloom filters from parquet files can be disabled using the catalog configuration property `parquet.use-bloom-filter` or the catalog session property `parquet_use_bloom_filter`. ({issue}`9471`)

@cla-bot cla-bot Bot added the cla-signed label Apr 23, 2023
@github-actions github-actions Bot added docs hive Hive connector iceberg Iceberg connector tests:hive labels Apr 23, 2023
@groupcache4321 groupcache4321 force-pushed the iceberg_bloomfilter_support branch from 6db0732 to 0f62687 Compare April 23, 2023 17:41
@groupcache4321 groupcache4321 changed the title Iceberg bloomfilter support Enable reading Parquet's bloomfilter statistics for hive connector Apr 23, 2023
@groupcache4321 groupcache4321 marked this pull request as ready for review April 23, 2023 18:02
@groupcache4321 groupcache4321 changed the title Enable reading Parquet's bloomfilter statistics for hive connector Enable reading Parquet's bloomfilter statistics for Iceberg connector Apr 23, 2023
raunaqmorarka
raunaqmorarka reviewed Apr 24, 2023
View reviewed changes
Comment thread lib/trino-parquet/src/main/java/io/trino/parquet/BloomFilterStore.java Outdated
Comment thread ...n/trino-hive/src/test/java/io/trino/plugin/hive/parquet/TestHiveParquetWithBloomFilters.java Outdated
Comment thread ...n/trino-hive/src/test/java/io/trino/plugin/hive/parquet/TestHiveParquetWithBloomFilters.java Outdated
Comment thread testing/trino-testing/src/main/java/io/trino/testing/BaseTestParquetWithBloomFilters.java Outdated
Comment thread testing/trino-testing/src/main/java/io/trino/testing/BaseTestParquetWithBloomFilters.java Outdated
Comment thread testing/trino-testing/src/main/java/io/trino/testing/BaseTestParquetWithBloomFilters.java Outdated
Comment thread .../trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergParquetWithBloomFilters.java Outdated
Comment thread docs/src/main/sphinx/connector/iceberg.rst Outdated
@groupcache4321 groupcache4321 force-pushed the iceberg_bloomfilter_support branch from 0f62687 to b1d730e Compare April 24, 2023 12:25
@raunaqmorarka raunaqmorarka merged commit 98f3281 into trinodb:master Apr 25, 2023
@raunaqmorarka raunaqmorarka mentioned this pull request Apr 25, 2023
Closed
@github-actions github-actions Bot added this to the 415 milestone Apr 25, 2023
@colebow colebow mentioned this pull request Apr 26, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@raunaqmorarka raunaqmorarka raunaqmorarka left review comments

@ebyhr ebyhr Awaiting requested review from ebyhr

@findinpath findinpath Awaiting requested review from findinpath

Assignees

No one assigned

Labels

cla-signed docs hive Hive connector iceberg Iceberg connector performance

Milestone

415

Development

Successfully merging this pull request may close these issues.

2 participants

@groupcache4321 @raunaqmorarka